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1. Introduction 

A one-dimensional Markov chain is defined by a one-sided, directional conditional 
independence property, and a process is Markov in the forward direction if and 
only if it is Markov in the backward direction. In two and higher dimensions, this 
property is replaced with a conditional independence property that is not associated 
with a particular direction. This leads to the notion of a Markov random field 
(MRF). 

Of course, the definition of MRF makes sense in one dimension as well (see 
Section [5]), but here the conditional independence is a two-sided property, and this 
is not the same as the Markov chain property. It is well known that any one- 
dimensional Markov chain is an MRF. However, the converse is not true: there 
are counter-examples for non-stationary, finite-valued processes and for stationary 
countable-valued processes. The converse does hold (and this is well known) for 
finite-valued stationary MRF's that either have full support or satisfy a certain 
mixing condition. In this paper we show that any one-dimensional stationary, 
finite-valued MRF is a Markov chain, without any mixing condition or condition 
on the support. 

Our proof makes use of two properties of the support X of a finite-valued sta- 
tionary MRF: 1) X is non-wandering (this is a property of the support of any 
finite- valued stationary process) and 2) X is a topological Markov field (TMF) (de- 
fined in Section [2]). The latter is a new property that sits in between the classes 
of shifts of finite type and sofic shifts, which are well-known objects of study in 
symbolic dynamics [5]. Here, we develop the TMF property in one dimension, and 
we will develop this property in higher dimensions in a future paper. 

While we are mainly interested in discrete-time finite- valued stationary MRF's, 
in Section [5] we also consider continuous-time, finite- valued stationary MRF's, and 
show that these are (continuous-time) Markov chains as well. 



2. Background 

2.1. Basic Probabilistic Concepts. Except for Section [5] (where we consider 
continuous-time processes) by a stochastic process we mean a discrete-time, finite- 
valued process defined by a probability measure /x on the measurable space (S^, B), 
where S is a finite set (the alphabet) and B is the product Borel cr-algebra, which 
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is generated by the cylinder sets 

[ai, . . . , an]j ■■= {x e : Xk+j = afe for A; = 1, . . . , n}, 

for any € S and j G Z. 

Throughout this paper, we will often use the shorthand notation: 

lj{ai,, . . . ,a,J = ii{{x eT,'^ : a;j^ = a^^ for A; = 1, n}) 

and similarly for conditional measure: 

//(a^i , . . . , I bj^,...,bj^) 

= iJ,{{x e : Xi^^ = ttt^, k = 1 . . .n} \ {xj^ =bj^, £ = 1 .. . m}). 

In particular, 

/x(ai, . . . , a„) = /i([ai, . . . , a„]o). 

Also, for X e and a < b e Z, we define 3T[a^5] — -^a^a+i • * • *^6* 
A stochastic process is stationary if it satisfies 

/i(ai, a2, . . . , a„) = /j-(a_,_|_i, aj-|-2j • ■ • ) cij+n) 

for all j e Z. 

A Markov chain is a stochastic process which satisfies the usual Markov condi- 
tion: 

/i(ao, ■■■,a„ I a_jv, • • • , a_i) = /Lt(ao, . . . , a„ | a_i) 
whenever /i(a_Ar, . . . , a_i) > 0. 

A Markov random field (MRF) is a stochastic process /i which satisfies 

/i(ao, . . . , a„ I a_Ar, . . . , a_i, o„+i, . . . , an+m) 
= IJ'iao, ■■■■an I a_i,a„+i), 
whenever ti{a-N, • • • , a_i, a„+i, . . . , a„+M) > 0. 

Note that the Markov chains that wc have defined hero first order Markov 
chains. Correspondingly, our MRF's are first order in a two-sided sense. One 
can consider higher order Markov chains and MRF's, but these can be naturally 
receded to first order processes, and all of our results easily carry over to higher 
order processes. 

More generally, a Markov random field can be defined on an undirected graph 

Q = (y, E), where V is a countable set of vertices and E, the set of edges, is a set 
of unordered pairs of distinct vertices. Specifically, an MRF on ^/ is a probability 
measure /it on for which ^([aj?] | [bdp] H [cg]) = tJ^iiap] \ [bap]) whenever G and 
F are finite subsets oi V, G D F — 9 and /i([6aF] H [cg]) > 0; here, the notation 
such as [ap] means a configuration of letters from S on the subset F CV, and dF 
(the boundary of F) denotes the set of v € V\F such that {u, v} € E for some 
u € F. It is not hard to sec that when the graph is the one-dimensional integer 
lattice, this agrees with the definition of MRF given above. 

The following is well known. We give a proof for completeness. 

Proposition 2.1. Any Markov chain (stationary or not) is an MRF. 

Proof. 

/i(ao, . . . , Un, Un+l, • • . , ttn+M \ d-N, ■ ■ ■ , d-l) 

= /z(a„+i, . . . , a„+M I 0,-N, ■ ■ ■ , a-i)M(<^Oj | ci-iv, • • • , a„+i, . . . , Un+M) 

Thus, 

fi{ao, . . . ,an I a-N, ■ ■ ■, a-i, fln+i, . . . , Ow+m) 
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_ fJ-{aO: ■ ■ ■ ,an,an+l, ■ ■ ■ ,a,n+M \ fl- AT , . . . , a_ i ) 
^J■{an+l, ■ ■ ■ ,an+M I a_Ar, . . . , a_i) 
_ At(ao, ■ ■ ■ 1 Qrt; flrt+l, ■ ■ ■ 1 Qrt+Af | Q-l) 
f^i^n+l, ■ ■ ■ ,an+M I O-l) 

the latter by Markovity. Since the last expression does not involve U-n, ■ ■ ■ ,a-2, 
we have 

n{ao, ■ ■ . ,an I a-N, ■ ■ ■ , o-i, On+i, ■ • • , an+Af) = l^i^o, . . . , a„ | a_i, a„_|-i, . . . , a„+M) 
Since the reverse of a Markov process is Markov, we then have, by symmetry, 

n{ao, ■ ■ ■ ,an I a_i, an+i, ■ ■ ■ , On+A/) = m(q^o, ■ • ■ , | O-l, On+l) 
Combining the previous two equations, we see that the MRF property holds. □ 

See 121 Corollary 11.33] for an example of a fully supported stationary MRF on 
a countable alphabet that is not a Markov chain. It is easy to construct examples 
of non-stationary MRF's that are not Markov chains (see the remarks immediately 
following Proposition l3.6p . 

2.2. Symbolic Dynamics. In this section, we review concepts from symbolic dy- 
namics. For more details, the reader may consult 5, Chapters 1-4]. 

Let E* denote the collection of words of finite length over a finite alphabet E. 
For w G E*, let \w\ denote the length of w. 

Let a denote the shift map, which acts on a bi-infinite sequence x G E^ by 
shifting all symbols to the left, i.e., 

((T{x))n = Xn+i for aU 71. 

A subset X of E^ is shift-invariant if a{x) G X for all x E X . A suhshift or a shift 
space X C E^ is a shift-invariant set which is closed with respect to the product 
topology on E^. Note that E^ itself is a shift space and is known as the full shift. 
There is an equivalent way of defining shift spaces by using forbidden blocks: for a 
subset F of E*, define 

Xjr = {x c^Y?- \ {xiXi+1 . . . Xi+j) i T for alH G Z and j G N U {0}}. 

Then X is a shift space iff X = Xjr for some T . 
The language of a shift space X is 

oo 

B{X) = y S„(X) 

n=l 

where 

Bn{X) = {w G E* : 3x e X s.t. (xi . . . x„) = w}. 

A sliding block code is a continuous map 4> from one shift space X , with alphabet 
E, to another Y , with alphabet E', which commutes with the shift: (poa = cro0. The 
terminology comes from the Curtis-Lyndon- Hedlund Theorem which characterizes 
continuous shift-commuting maps as generated by finite block codes, namely: there 
exist m,n and a map $ : B^+n+iiX) — y E' such that 

{(f>ix))i = ^{xi^rnXi-m+l ■ ■ ■ X^+ri) 

li m — — n, then (jj is called a 1-block map. A conjugacy is a bijective sliding 
block code, and a factor map or (factor code) is a surjective sliding block code. 

3 



Figure 1 . golden mean shift 




A shift of finite type (SFT) is a shift space X — Xjr where F can be chosen finite. 
An SFT is called k-step if k is the smallest positive integer such that X = Xjr and 
J" C Yj^+'^. a 1-step SFT is called a topological Markov chain (TMC). Note that 
a TMC can be characterized as the set of all bi-infinite vertex sequences on the 
directed graph with vertex set E and an edge from x to y \S xy ^ J- . TMC's 
were originally defined as analogues of first-order Markov chains, where only the 
transitions with strictly positive transition probability are prescribed [7]- 

The most famous TMC is the golden mean shift defined over the binary alpha- 
bet by forbidding the appearance of adjacent f's, equivalently X — S^n}. The 
corresponding graph is shown in Figure [TJ 

Just as MRF's and Markov chains can be recoded to first order processes, any 
SFT can be recoded to a TMC; more precisely, any SFT is conjugate to a TMC. 

A shift space X is non-wandering if whenever u € B{X), there exists a word v 
such that uvu G B{X). The support of a stationary process /i on is the set 

supp(/^) = E^\ U H„, 

where A/'(/i) is the collection of all cylinder sets with /i([a]„) = 0. TMC's are exactly 
the set of shift-invariant sets that can be the support of a (first-order) Markov chain. 
The reason for our interest in the non- wandering property is that the support of any 
stationary measure is non-wandering; this follows immediately from the Poincare 
Recurrence Theorem (see 0). 

A shift space is irreducible if whenever u,v G B{X), there exists w such that 
uwv G B{X). Clearly any irreducible shift space is non- wandering. By using the 
decomposition of a non-negative matrix into irreducible components, it is easy to 
see that a TMC is non-wandering iff it is the union of finitely many irreducible 
TMC's on disjoint alphabets. For a subshift X, the period of x € AT is min{i G 
N I (t'^{x) = x}. The period of X is defined as the greatest common divisor of the 
periods of elements of X. For any irreducible TMC of period p, one can partition S 
into So, Si, ... , Sp^i such that if x £ X , xq G S^ then Xi G '^i+j (mod p)- The Sj's 
are called the cyclically moving subsets. 

In the full shift, any symbol can appear immediately after any other symbol. 
Irreducible TMC's have a related property, described as follows. The index of 
primitivity of a TMC X with period p is the smallest positive integer t such that 
for any a G Si,6 G Sj, there exists x G X such that xi = a,Xtp+j-i+i = b. 
Every irreducible TMC has a finite index of primitivity. The terminology comes 
from matrix theory where, for a primitive matrix A, the index of primitivity is 
understood to be the smallest positive integer such that is strictly positive, 
entry by entry. For example, the golden mean shift is irreducible, its period is 1 
and its index of primitivity is 2. 

A Sofia shift is a shift space which is a factor of an SFT. By a recoding argument, 
it can be proved that any sofic shift A" is a 1-block factor map of a TMC. This 
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Figure 2 . even shift 




amounts to saying that a shift space X is sofic iff there is a finite directed graph, 
whose vertices are labelled by some finite alphabet, such that X is the set of all label 
sequences of bi- infinite walks on the graph. Such a labelling is called a presentation. 
The most famous sofic shift is the even shift defined by forbidding sequences of the 
form fO^""*"^! for all n G N. A presentation is shown in Figure [5] 

The following is one of the many useful characterizations of sofic shifts. Let X 
be a shift space. For all w e B{X), the follower set of w is defined as 

F{w)^{yeB{X) : wy e B{X)}. 

The collection of follower sets is denoted by 

F{X) = {F{w) : w e B{X)}. 

A shift space X is sofic if and only if F{X) is finite. Similarly, one can define 
predecessor sets: P{w) — {y £ B{X) : yw G B{X)}, and P{X) denotes the 
collection of all predecessor sets. A shift space X is sofic if and only if P{X) is 
finite. 

One can also work with follower sets of left-infinite sequences. Let Boo {X) denote 
the set of all left- infinite sequences x~ for which there exists a right-infinite sequence 
such that x~x~^ G X. Let 

Foo{x-) = {y e B{X) : x'y E Boo{X)}. 

Note that Foo{x^) is the decreasing intersection of It follows that 

X is sofic iff the collection of all -^00(2;") is finite [4, Lemma 2.1]. 

Finally, we state a useful result that is probably well known, but we do not know 
of an explicit reference. So, we give a proof for completeness. 

Lemma 2.2. Let X be a shift space. If X has dense periodic points, then X is 
non-wandering. The converse is true if X is sofic. 

Proof. Assume that the periodic points are dense in X. Let u G Bn{X) for some 
n G N. Then there exists a periodic point x E X such that „] — u. Since x is 
periodic, there exists r G N and a word v such that x\^i r] = uvu. Thus uvu G B{X), 
and so X is non- wandering. 

For the converse, assume that X is sofic and is non-wandering. Let F be a TMC 
and : Y — > X a f-block factor map. 

Consider a word u G Bn{X) for some n G N. Since X is non- wandering, for 
any M, there exist vi . . . vm G B{X) such that w — uviuv2 ■ . ■ vmu G B{X). Let 
M > \Bn{Y)\. Take z G X such that for some interval /, zj = w. Then there exist 
ri < r2 < . . . < rM such that zj^.^ ^t+n-i] — " for t — Ij • • ■ j M. Let y € Y such 
that 4>{y) z. Since M > \Bn{Y)\ we can find h < k eN such that 

y[rh,rh+n-l] = y[rk,rk+n-l]- 
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Consider y' defined by 

and y[ = y^+r,-r,. 
for all t £ Z. Clearly y' is periodic. And y' <E Y since F is a TMC. Also, 

Since u £ B{X) was arbitrary, this proves that periodic points are dense in X. □ 

3. Topological Markov Fields 

Definition 3.1. A shift space X C T,^ is a topological Markov field (TMF) if 
whenever V, X 'S, |z| = and uvwxy,vzx ^ B{X), then uvzxy d B{X). 

We have defined TMF's as "1-step" objects, in that v, x are required to be letters 
of the alphabet, i.e., words of length one. One can naturally extend the definition 
to "fc-step" objects, by requiring v,x to be words of the same length fc, and results 
of this section extend easily to this class. 

The defining property for TMF's is equivalent to another property, which appears 
stronger, and is suitable for generalization to higher dimensions. For this, recall 
that the boundary of C, denoted dC, denotes the set of integers in Z \ C that are 
adjacent to an element of C. 

Proposition 3.2. A shift space X is a TMF if and only if it satisfies the following 
condition: 

For all x,y G X and finite C G such that x — y on dC, the point z £ 
defined by 

_ jx onCUdC 
^ ~ on {CUdCy 

belongs to X. 

Proof. The "if" direction is trivial. For the "only if" direction, we use the fact that 
any finite subset of Z is a disjoint union of finitely many intervals (of integers). If 
C consists of only one interval, then we get the condition immediately from the 
definition of TMF. Now, proceed by induction on the number of intervals. □ 

The reason for our interest in TMF's is the following simple result. 

Lemma 3.3. The support of a stationary MRF is a TMF. 

Proof. Let /x be an MRF, X = supp(/i) and uvwxy,vzx £ B{X) with \z\ = \w\ and 
v,x gTj. By definition of X, ^(uvwxy) > 0, and so /i([ww]o n [xy]^^^^^) > 0. Since 
^ is an MRF, 

(1) i^{[z]\uv\ I [Ho n [xy]\uvw\) = I Hi«i n 

Since by the definition of X, fi{vzx) > 0, it follows that the right hand side of ([T]) 
is positive. Thus ^{uvzxy) > 0, and so uvzxy £ B{X) as desired. □ 

For a shift space X and w £ B{X), let 

C{w) = {{x,y) : xwy £ B{X)} and C{X) = {C{w) : w £ B{X)}. 

The following is a simple restatement of the definition of TMF in terms of the sets 
CM: 
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Proposition 3.4. A shift space X is a TMF iff for all n e N and for all w,u e 

Bn{X), 



(2) (wi = ui and w„ — u„) ^ C{w) — C{u). 

Proposition 3.5. Any TMF is sofic. 

Proof. Let X C be a shift space that is not sofic. We wih prove that X is not 
a TMF. 

Since X is not sofic, there are infinitely many left-infinite sequences with distinct 
follower sets. Thus there exist distinct left-infinite sequences w^, w^, . . . , zx;'^! """^ 
with distinct follower sets. Note 

and i^((w*)[_„_i,_i]) C i^((wO[-„,-i]) for all 1 < i < -h 1 and n e N. If, 
for each n, the F{{w^)[_„ _^) are not distinct, then there exist ii ^ J2 such that 
Foo{w'^'^) = i^oo(w'^), contradicting the assumption. Therefore there exists an uq G 
N such that the F((w*)[_„^ _i]) are all distinct. Hence we can choose u, u' € Bng(X) 
such that ui = u[ and Un„ = u'^-^^ but F{u) ^ F{u'). Hence there exists b e B{X) 
such that exactly one of ub and u'b is an element of B{X). It follows that there 
exists a G B{X) such that exactly one of aub and au'b is an element of B{X). 
Therefore C{u) ^ C[u'). By Proposition |33 X is not a TMF. 

□ 

It is clear that any TMC is a TMF. The following example shows that a TMF 
need not be a TMC. In fact, this TMF is not even an SFT. This is an elaboration 
of an example given in [Tj. Let 

Xnot ^ {0°°, 0°°1°°, 1°°, 1°°02°°, 2°°}; 

to clarify the notation, l°°02°° refers to the point x such that 

{1 if i < 
if i = 
2 if i > 

and all its shifts. 

Proposition 3.6. X„ot is a TMF but not an SFT (and in particular is not a 
TMC). 

Proof Xnot is not an SFT since for all n, 01", 1"0 G B{Xnot) but 01"0 ^ B{Xnot). 

We will check that the restriction of any configuration on the positions and 
n + 1 uniquely determines the configuration on either [1,7t.] or [0, n -f- 1]"^. This 
clearly implies condition ([2|) and thus, by Proposition 13. 4[ Xnot is a TMC. 

To see this, first observe that for n > 2, 

Bn{Xnot) = {0",0'=l""'',l",l'"02"-'"-\l"-i0,02"-\2" |0<A:<n, 0<77i< 

Among these, the only pairs of distinct words with the same length that begin 
and end with the same symbol are q'^'I"-'^-, O''' 1"~'=' and l'"02"-"-\ i™'o2"-'"'-i. 

7 



Now, observe that 

C(0'=l"-'=) = C(0'^'l"-'^-') ={(0',P)} 
C(l"02"-'"-^) = (7(1™' 02"-"'-!) ={(!', 2^)} 
for < k,k' < n and < m, m' < n — 1. 

□ 

Since Xnot is countable, any strictly positive countable probability vector defines 
a measure whose support is Xnot- Any such measure is an MRF because any 
valid configuration in Xnot on the positions and n + I uniquely determines the 
configuration on [l,n] or [0,71 + 1]^^. Thus, there exist non-stationary finite- valued 
MRF's which are not Markov chains of any order. 

Now we will prove that there is a finite procedure for checking whether a sofic 
shift is a TMF. The following characterisation of sofic shifts will be used. 

Proposition 3.7. A shift space X is sofic iff \C{X)\ < oo. 

Proof "If:" For each F e F{X), fix some wp e B{X) such that F{wf) = F, Now, 
consider the map : F{X) ^ C{X), defined by C{wf). li F ^ F' , then 

F{wf) 7^ F{wF') and thus C{wf) ^ C{wf')- Thus, is 1-1, and so F{X) is finite. 

"Only If:" For a follower set F <E F{X) and a word w e F, let Fyj — {y : wy e 
F}. Note that F^, is a follower set. We claim that 

(3) C{w) — {{x,y) : there exists F s.t. w & F,y & F^,, and x G C\zeFP{z)}. 

To see this, first note that if {x,y) G C{w)^ then wy E F = F{x). Then w E F, 
y G -F^ and for all z G F = F{x), xz G B{X), and so a; G P(z)- 

Conversely, if w G F and y G F[u, then z — wy G F. If x G n2gF-P(^), then 
taking z = wy, we see that xwy G -B(X), and so {x,y) G C(w). This establishes 
©. 

By ([3]), we see that C{w) is uniquely determined by the set 

{(nzeF-P(^), ^jii) : F is a follower set that contains w}. 

Thus, |C(X)| is upper bounded by the number of functions whose domain is a 
subset of F{X) and whose range is subset of 2-^^-^^ x F{X) and is therefore finite 
(here, for a given word w, the domain D is the collection of follower sets that 
contain w and the function is: for F E D, g{F) = {{P{z)}zeFTF.w)). 

□ 

Now we can introduce the procedure. 

Theorem 3.8. There is a finite algorithm to check whether a given sofic shifi X is a 
TMF (here, the input to the algorithm is a labelled finite directed graph presentation 
ofX). 

Proof. By combining the next two lemmas, we will see that condition ([2]) can be 
decided by checking words of a length bounded by an explicit function of a presen- 
tation of X. 

Lemma 3.9. Let n > \C{X)\'^ and w,u E Bn{X) such that wi = ui and Wn = Un- 
Then there exists r < \C{X)\'^, w*,u* G Br{X) such that w* = u*, w* — u*, 

C{w*) = C{w) and C{u*) = C{u). 
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Proof. We claim tliat if C(a) = C{c) for some a, c G B{X) tlien for any h G B{X) 
such that a6,c& G B{X), C{ab) = C{cb). To see this, observe: 



C(a6) = 



fey G B{x)} 



{x,by)eCia)} 
{x,by)eCic)} 



= {(x^y) 
= {ix,y) 
= C{cb). 

Consider the set {{C{u'),C{w'))} such that u' and w' are proper prefixes of u 
and w (respectively) and of the same length. This set has size at most |C(X)p. 
Since n > |C(X)|^, there are distinct pairs {u',w') and {u",w"), where u' and u" 
are prefixes of u, w' and w" are prefixes of w such that 

\u'\ = \w'l \u"\ = \w"\, Ciu') = CK) = CK'). 

We may assume that < \u"\. 

Define words ft, fc by: u — u"h and w = w"k. Since 

C(m') = C{u") and C{w') = 

we have 

C{u'h) = C(u"ft) = C(w) and C{w'k) = C(w"fc) = C(w). 

If u* = u'h and w* — w'k have length at most |C(X)|^, we are done; if not, 
inductively apply the same argument to u*, w* instead of m, w. □ 

Let 

Cmiw) = {{x,y) : xwy e B{X),\x\,\y\ < m}. 

Lemma 3.10. Let m = max{|P(X)|, |F(X)|} and w,u ^ B{X) such that \w\ — \u\, 
Wi = ui and w^^^ = Then 

c{w) = c{u) iff CM = CM- 

Proof. Assume that Cm{w) — Cm{u). Assume that awb G B{X). It suffices to 
prove aub G B{X). 

If \(A^ 1^1 — then there is nothing to prove. Suppose instead \a\ > m. By the 
choice of m, there exists 1 < i < i' < |a| such that 

F(ai...ai) = F{ai...ai>). 

Let a' = ai . . . a^ai'+i . . . ajoj. Then a'wb G B{X). And aub G B(X) iff a'ub G 
B{X). Since \a'\ < |a|, by induction on \a\, we can assume \a'\ < m. By a similar 
argument, we can find b' such that \b'\ < to, a'wb' G B{X). And aub G B{X) iff 
a'u5' G B{X). Since a„(w) = C™(m), a'ub' G B(A:). Therefore, au6 G B{X), as 
desired. □ 

Proof of Theorem [M' By Lemmas [31] and [XTUl AT is a TMF if and only if 
for all r < |C(A)p, w,u G B^^X) such that Wi — ui and = and all 
TO = max{|P(A)|, |_F(A)|}, Cm{w) = Cm{u). This reduces to the problem of 
deciding for aU words z of length at most |C(A)p+2max{|P(A)|, |F(A)|}, whether 
z G B{X). This is accomplished by using a labelled graph presentation of a sofic 
shift 5, Section 3.4]; such a presentation can be used to give a bound on |F(A)|, 
\P{X)\, and therefore also on |C(A)| by a bound given implicitly at the end of the 
proof of Proposition 13.71 □ 



For any sofic shift X, we can endow C{X) U {*}, where -k is an extra symbol, 
with a semigroup structure: 

C{w)C{u) — C{wu) ii wu G B{X) and * otherwise 

One can show that the multipUcation is well-defined and formulate an algorithm in 
terms of the semigroup to decide if X is a TMF. This is consistent with the spirit 
in which sofic shifts were originally defined |10j . 

Theorem 3.11. Let X be a shift space. The following are equivalent, 
(a) X is the support of a stationary Markov chain, 
(h) X is the support of a stationary MRF. 

(c) X is non-wandering and a TMF. 

(d) X X X is non-wandering and X is a TMF. 

(e) X is a TMC consisting of a finite union of irreducible TMC's with disjoint 
alphabets. 

Proof, (jg) implies ([b| by Proposition l2.11 and (|b| implies (jg) follows from Lemma [X51 
and the fact, mentioned above, that the support of a stationary probability measure 
is non- wandering. 

For (jcj) implies (|d|, first observe that, by Proposition [331 ^ is sofic, and then by 
Lemma [2.21 X is non- wandering iff X has dense periodic points. Then follows 
since X has dense periodic points iff X x X has dense periodic points. 

We now show that |ej) implies (jaj) . Any irreducible TMC is the support of a sta- 
tionary Markov chain, defined by an irreducible (stochastic) probability transition 
matrix P such that P^y > iff a;?/ G B2{X). 

Finally, we show that (|d]) implies (jej). Let u,w ^ B{X) and u G S such that 
uv G B{X) and vw G B{X). To prove that X is a TMC it is sufficient to show that 
uvw G B{X). 

There exists w G B{X) with = \w\ and uvw G B{X). Also, there exists 
u G B{X) such that \u\ — \u\ and uvw G B{X). Since X x X is non-wandering, 
there are words x,y € B{X) with |a;| = \y\ and Hvwxiivw, uvwyuvw G B{X). Since 
AT is a TMF, uvwxuvw G B{X). In particular, uvw G B{X), and so AT is a TMC. It 
is easy to see that if X x A" is non- wandering, so is AT. Thus, AT is a non- wandering 
TMC. As mentioned in Section [2T2| it follows that X is a finite union of irreducible 
TMC's with disjoint alphabets. □ 

We remark that in each of (jc|) and (|d|, TMF can be replaced with TMC. Also, as 
mentioned above, a shift space X is nonwandering if A" x AT is nonwandering. The 
converse is true for TMF's due to the equivalence of (jg) and (|d|; in fact, the converse 
is true more generally for sofic shifts (this follows from Lemma [221). However, the 
converse is false for shift spaces in general. 

4. Stationary MRF's are Markov chains 

Theorem 4.1. Let ^ be a stationary measure on S^. Then fi is an MRF iff it is 
a Markov chain. 

Proof. By Proposition [^Hl every Markov chain is an MRF. 

For the converse, let ^ be an MRF. By Theorem 13. Hi supp{ii) is a finite union 
of irreducible TMC's on disjoint alphabets. Therefore ^ is a convex combination of 
MRF's supported on irreducible TMC's. So, it suffices to assume that X = supp{fj,) 
is an irreducible TMC. 
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: X° ^ {B"^{X)}^ 



If X were a full shift, then it would be a Markov chain by ^ Corollary 3.9]. We 
will reduce to this case. 

Let p be the period and t be the index of primitivity of X. Let Eq: ^ii • ■ • : Ep_i 
be the cyclically moving subsets of X. For r e N, a multiple of p, let 

B°{X) = {a^r . . . a-i e Br{X) : a^r e Eq}- 

Fix • ■ • 2^-1 G ^r(^): ^^o £ Eq and i a multiple of p such that L > r + tp. Fix 
i e N. Then, with the summations below taken over all values of aiL^r ■ ■ ■ o-iL-i £ 
B°{X), we have 

^(a;o|x_l, . . . ,X-r) = ^ fJ.{xo, aiL-r, ■ ■■ , aiL-l | X-i, . . . , X-r) 
= ^ /i(a;o I X-l, . .., X^r, CliL-r, • . • , aiL~l)^J.{aiL~r, • • • , fljL-l I a;_l, . . . , X-r) 
= ^ ^(Xo I X-l,atL-r)f^{aiL-r, • • • , OiL-l | X-l, . . . , X-r) 

(the last equality follows from the MRF property). 
Thus, 

(4) 

H{xo\X-i,. . .,X-r) = ^ Hixo I X-i,aiL-r)lJ'{aiL-r, ■ ■ ■ , OiL-l | X-l, . . . ,X-r). 

Let 

and let 
be given by 

{(t>{x))i = {XiL-r ■ .-XiL-l). 

Let ^' be the probability measure on {B^{X)}^ given by the push- forward of the 
measure ^ by 0, i.e., fi'{U) — ([/)). 

Observe that supp(/x') ~ {i?,J?(X)}^ by definition of L and We show that fi' is 
a stationary MRF with alphabet B^{X), as follows. Let x G Xq. Below, we write 
XiL-r ■ ■ ■ XiL-i as xl^Zr wheu viewed as a word of length r and as bi when viewed 
as a a single symbol in the alphabet, B^{X), of /i'. Then 

^'([5o, . . . ,6„]o I [b-N, ■ ■ ■ ,b-l]-N [bn+l,---bn+M]n+l) 

_ f -1 nL-1 I -7VL-1 -L-1 (n+l)L-l (n+Af)L-lN 

— ^[X_,j,, . . . , X^j^_^. I X_j^]^_^, . . . , X_^_^, 2;(„+l)L-r' • ■ • ' {n+M)L-r> 

^ li{xZl,.-.,x'^'lZl I a;-L-i,a;(„+i)L-r) 

_ ^ -1 nL-1 I -L-1 (n+l)L-lN 

— fl{X_^, . . . , X^j^_^ I ^-L-r' ^(n+l)L-r/ 

= A^'([6o, • • -^bnh I n [6„+i]„+i). 

Thus, /i' is a stationary MRF with full support. 

By 2,, Corollary 3.9], /i' is a fully supported stationary Markov chain, and thus 
has a positive stationary distribution, which we denote by tt. 

Therefore, for any a-r ■ ■ ■ a_i G B'^{X), we have 

lim fJ,{aiL-r = a-r, . . . , liL-l = O-l | X-i, . . . , X-r) 

2— ^OO 

= lim = a_r • • • O-i I ^0 — X-i ■ ■ ■ X-r) = 7r(a_r • ■ • O-i)- 

2— >00 
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By compactness of [0, 1], there is a sequence ik such that 

k—^oo 

exists. Returning to (|4]), we obtain 

fi{xQ I x^i,x-2, ■ ■ ■,x-r) = lim y^u{xa \ x-i,ai^L-r)f^{aii,L-r, ■ ■ • ,aifeL-i | x^i,x 
= 7r(a_r • • -fl-i) lim /^(xq I x_i,ai^i_r), 

a_r...a_ieBO(X) 

which does not depend on any of x_2, a;_3, . . . , X-r . Therefore, 
n{xo I a;_i, a;_2, . . . , x^r) = ^(a^o I a^-i)- 

Since the only restriction on r is that it is a multiple of p, it can be chosen arbitrarily 
large and so /i is a stationary Markov chain, as desired. □ 

As noted in the course of the proof of Theorem l4.11 in the case where the support 
is a full shift, the result is well known. It can also be inferred from [31 Theorems 
10.25, 10.35] in the case where fj, satisfies a mixing condition [21 Definition 10.23] 
(that condition is slightly stronger than the concept of irreducibility used in our 
paper; the results in l2] apply to certain stationary processes that may be infinitely- 
valued) . 

Closely related to the notion of MRF is the notion of Gibbs measures [Qj . It is 
an old result [S] that any Gibbs measure defined by a nearest-neighbour potential 
is an MRF. Also it is easy to see that any Markov chain is a Gibbs measure, defined 
by a nearest-neighbour potential determined by its transition probabilities. So, in 
the one-dimensional (discrete-time, finite-valued) stationary case, MRF's, Gibbs 
measures and Markov chains are all the same (where we assume that our MRF's 
and Markov chains are first order and our Gibbs measures are nearest- neighbour) . 

5. Continuous-time Markov random fields and Markov Chains 

We begin with a definition of continuous-time processes. This presentation is 
compatible with standard references, for example (Gj Section 2.2]. To avoid ambi- 
guity and measurability issues, we will use the following: 

Definition 5.1. A continuous-time stationary process (CTSP) taking values in a 
finite set T, is a translation-invariant probability measure on the space i?C(S) of 
right- continuous functions from R to E with the a-algebra B generated by cylinder 
sets of the form: 

[ai, . . . ,an]ti,...^t„ = {x e RC{T,) : Xi, = Oi , i = 1, . . . , n}, 

where G S and ti £ Q. 

For sets A,B £ B and Borel measurable / C R, we will use the shorthand 
notation fi{A\B and xj) to mean fi{A\B n {y £ RC{J^) : yi = xj}). 

Definition 5.2. A continuous-time stationary Markov random field (CTMRF) 
taking values in a finite set Y, is a CTSP which satisfies: 

;u([ai, . . . ,a„]ii^...,i„ I [a, 6]s,t) = /x([ai,...,a„]fi,...,f„ | [a, &]s,t n &m]si,. ..,«„) 

whenever 

ti,...,tn e {s,t), Si,...,Sm G [s,tY and ^i{[a,b]s,t H [bi, . . . ,bm]si,...,s^) > 



Informally, a CTMRF can be viewed as a stationary process such that for all 
s <t, the distribution of {xu)ue{s,t) is independent of {xu)ue[s.t]<: given {xs,xt}- 

Definition 5.3. A continuous-time stationary Markov Chain (CTMC) taking val- 
ues in a finite set T, is a CTSP which satisfies 

^([ai,...,a„]ti,...,t„ I [a]s) = ^ ([ai, . . . ,a„]tj^...,t„ | [a]s H [5i, . . . , 6„]si,...,s„J , 

whenever ii, . . . , i„ > s, Si, . . . , Sm < s and /i([a]s n [5i, . . . , fom]si,...,s„) > 0. 

In this section, we prove: 

Proposition 5.4. Any continuous-time (finite-valued) stationary ergodic Markov 
random field is a continuous-time stationary Markov chain. 

Note that in this statement, there are no positivity assumptions on the condi- 
tional probabilities /it ([ai, . . . , a„]tj_...,t„ | [a, 6]s,t)- 

We prove this result by reducing it to the discrete-time case. 

Proof. Let /i be a CTMRF. It suffices to show that for any n G N and t < ti < 
t2 < ... <tn 

fl{xt I Xt^,Xt2,...,XtJ ^ tJ,{xt I XtJ. 

Let e > 0. By the right continuity of the elements of i?C(E), there exists m E N 
such that 

(5) \n{xt\xt,,xt^,...,xtj- fi{xi\xi^,Xi^,...,XtJ\ < e 

(6) \fi{xt\xt,)- fi,ixi\xij\ < e 

where for any r S R, f means ^^^^ . Also, the process restricted to evenly spaced 
discrete points forms an MRF; that is, defining (f> : i?C(S) — > by 4>{x)i = xj_, 
the push-forward of /x, given by jl{F) = fi{(f)^^ {F)), is an (discrete-time) MRF. By 
Theorem 14. 1[ /l is a (discrete-time) Markov Chain. Hence 

fJ-i^i I ^U^^h^-'-^^iJ = P'{4>{x)mi I 4>{x)mh,(t){x)^^i^, . . . ,(t>{x)^lj 

= H4>{x)mi\4>{x)mt\) 

(7) = ^l{x^\x^^). 

By ([5]), ([B]), and the triangle inequality, we get 

\n{xt I Xt^,xt2,...,xtJ - fi{xt I xt^)\ < 2e. 

Since e was arbitrary, 

H{xt \xt^,xt2,---,xtj = n{xt I XfJ. 

□ 

We conclude this section with an ergodic-theoretic consequence. A measure- 
preserving flow is a collection of invertible measure preserving mappings {Tt}teR of 
a probability space such that Tf^g = Tt oTg for all s and the map (t, x) i— > Tt{x) 
is (jointly) measurable. Any CTSP is a measure preserving flow. 

A measure-preserving flow is ergodic if the only invariant functions are the con- 
stant functions, i.e., if for all t, f o Tt = f a.e., then / is constant a.e. And {Tt} is 
weak mixing (or weakly mixing) if it has no non-trivial eigenfunctions, i.e., if A G C 
and for all t, f oTt — A*/ a.e., then / is constant a.e. 
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Weak mixing is in general a much stronger condition than ergodicity. However, it 
is weh known that any stationary ergodic continuous-time Markov chain is weakly 
mixing 3 . So, a consequence of Proposition [213] is: 

Proposition 5.5. Any continuous-time (finite-valued) stationary ergodic Markov 
random field is weakly mixing. 

Below, we give a direct proof of this result. For / C M, we use to denote 

the tr-algebra generated by cylinder sets of the form: 

[ai,...,an]t^^...,t„ = {x e RC{Y.) : = , i = 1, . . . , n}, 

where G E and e Q fl /. 

Proof. Suppose /i is an ergodic CTMRF on the alphabet S. Suppose that / is 
a non-constant L^(/i)-eigenfunction of Tj, with A the corresponding eigenvalue: 
f{Tt{x)) = X* f{x). By the assumption that Tt is ergodic, A ^ 1. By normalizing, 
we can assume |/| = 1 a.e., thus II/II2 = 1- For any e > 0, there is a sufficiently 
large n and a S(E'^~"'"^)-measurable function /„ with ||/„ — /II2 < e. 

It follows that for any t e R, ||/„ — TjA^Vnlb < 2e. Now denote by /„ the 
conditional expectation of /„ with respect to By the MRF property 

of fi, since /„ is i3(E(~"'"))-measurable and TtA~*/n is S(S(~"+*'"+*))-measurable 
it follows that for t > 2n, 

J {fn){TtX-'fn)dfi^ J {fn){TtX-'fn)dpL. 

It follows that 

(8) \\fn-TtX-'fn\\2<2e. 

We claim that for sufficiently small e it is impossible for ([8]) to hold simultane- 
ously for all sufficiently large t. To see this, first note that due to the MRF property, 
/„ takes at most |Sp values. Thus, there is some c G C with |c| = 1 such that for 
all X, \fn{x) — c\ > 2\h\^' ^^'^ some d G C with \d\ — 1 with ij.{fn{x) = d) > -prp-. 
Now take t > 2n such that A*c? = c. 

Now for any x such that f"{x) — d, let z = Ttfn{x). Then 

\Ux) ~ X-%Ux)\ = \d- A-*z| = |A*d ^z\ = \c-z\> 

and so ||/„ — TtA^*/„||2 > 2|^- This contradicts © for e < j^r^ □ 

6. MRF's IN Higher Dimensions 

There is a vast literature on Markov random fields and Gibbs measures in higher 
dimensions. Most of the concepts in this paper can be generalized to higher di- 
mensional processes. For instance, we can define a Z'' TMF as a Z'^ shift space 
X such that whenever A and B are finite subsets of Z'' such that OA C B C A'^, 
and x,y ^ X such that xgA = VdA, then there exists z G X such that za = xa 
and zb = Vb- With this definition, the proof of Lemma 13.31 carries over to show 
that the support of a Z'' stationary MRF is a Z"^ TMF. However, most of the other 
results in this paper fail in higher dimensions. For example, there are Z'^ TMF's 
which are not even sofic. And there are stationary Z** MRF's that are not Gibbs 
measures. However, there are some positive things that can be said, and this is a 
topic of ongoing work. 
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